Decison Boundary and hyperbolic unfolding

Decision boundary of a neural network

Dataset with two classes

Train a neural network on the binary classification task

# define fully connected NN architecture
circle_detect = Net(0, [2,10,20])


# Train neural net on binary classification task
_ = train_classification_nn(circle_detect, data, label, n_epochs=15)

epoch	train_loss	valid_loss	accuracy	time
0	0.725640	0.717628	0.491000	00:00
1	0.676842	0.639662	0.491000	00:00
2	0.560278	0.509667	0.909000	00:00
3	0.466550	0.434938	0.926000	00:00
4	0.404259	0.381256	0.960000	00:00
5	0.353271	0.338769	1.000000	00:00
6	0.329343	0.324409	1.000000	00:00
7	0.320989	0.319070	1.000000	00:00
8	0.317565	0.316634	1.000000	00:00
9	0.315937	0.315435	1.000000	00:00
10	0.315094	0.314812	1.000000	00:00
11	0.314643	0.314483	1.000000	00:00
12	0.314412	0.314322	1.000000	00:00
13	0.314310	0.314262	1.000000	00:00
14	0.314281	0.314253	1.000000	00:00

Contour plot of the neural network function

The descion boundray is $\left\{x\in \mathbb R^2:\ nn(x)=\frac 12\right\}$, i.e. all points where the output of the neural network is ambiguous.

Ways to compute the decision boundary

First Approach: Sampling algorithm

• Using methods described in "On The Topological Expressive Power of Neural Networks", brought to our attention by Stefania at the last AppTop Meet-up.
• Implementation by Matteo.

Gradient flow method

• Pushing point to the decision boundary.
• Using automated differentiation for computation graphs of tensors.

Sample points $x_i$ with gradients $-\nabla_{x_i} \left(nn(x_i)-\frac 12\right)^2$.

After 50 iterations and a filtration with respect to $|nn(x_i)-0.5|>0.01$ we end up with the red points:

Comparision with the previously plotted decision boundary.

Hyperbolic unfolding

Entanglement problem for tori